The recent advances in CMOS sensor image (CIS) technology have allowed manufacturing of the high resolution image sensors which usually also contain basic image processing functions on the same chip as the image sensor array itself.
The most utilized sensing elements of today's CMOS image sensors comprise a pinned photodiode, a transfer gate, a floating diffusion, a reset gate, a source follower gate and a row select switch. These devices are usually referred to as 4T pixels (four transistor pixels). They combine the advantages of CCD image sensors of low noise and high sensitivity with the advantages of high integration provided by the CMOS technology. While 4T pixels excel in low signal range the physical limitations of the photodiode and floating diffusion limit the maximum achievable signal to a rather moderate value.
The average 4T pixel floating diffusion capacitance is in the range of several fF what typically translates into roughly 10000 electrons full well capacity. The dynamic range of the image sensor is defined as the ratio of the highest non-saturating signal to the noise in the absence of optical excitation. The typical 4T pixel dark noise is in the range of several electrons, thus the maximum dynamic range of a 4T pixel is usually limited to 60-70 dB.
The dynamic range of the imaged scenes can be quite high. The ability to discriminate objects in a high dynamic range scene is especially important in automotive driving assistance applications. A typical night driving scene would require a dynamic range of more than 100 dB to capture the dark and the bright details. Other examples of high dynamic scene can be a car entering a dark garage or a tunnel in the bright day.
Numerous approaches for dynamic range extension have been proposed in the prior art. A non-exhaustive list includes multiple frame recombination, autoreset pixel (U.S. Pat. No. 6,927,796), logarithmic response pixel (US2003/136915), multiple integration potential well pixel (US2009/021627), partial reset approaches (e.g. U.S. Pat. No. 7,586,523 and US2011/221944).
The proposed approaches for the image sensor dynamic range extension however have significant drawbacks: the multiple frame approaches suffer from high image lag and require a full frame buffer and a powerful processing unit to merge the low dynamic range frames into a high dynamic range image, the autoreset and multiple integration potential well pixels achieve the dynamic range extension at the price of more complex pixel circuitry, so killing the possibility to build practical high resolution image sensors.
The logarithmic response pixel described in US2003/136915 suffers from poor dark signal performance due to the usage of an n+−p substrate diode and high technological pixel to pixel mismatch of the non-linear elements resulting in a high fixed pattern noise of the wide dynamic range image. The per-pixel calibration is required to bring the noise to acceptable levels in practice. Yet another approach, e.g. described in US 2010/0224765, makes use of a non-linear device connected to the floating diffusion to preserve the low light performance of the 4T pinned diode pixels. The signal charge after filling the photodiode potential well leaks to the floating diffusion potential well and creates a highly non-linear voltage drop on the sublinear element connected to it. Another logarithmic response pixel is disclosed in US2008/0252762: the pixel signal readout is split into 2 phases of reading out the first linear signal and the second nonlinear signal caused by the nonlinear logarithmic voltage drop over a potential barrier. While potentially these last two approaches are capable of achieving a quite high saturation point and have the same low light noise as the linear 4T pinned diode, the pixel medium and high light noise performance of such imagers is bad due to the uncertainty in the potential at which the signal charge starts to leak to the photodiode and also due to technological mismatch of the non-linear elements. In practice it also requires quite high computational power to perform per-pixel calibration.
The most successful approach for extending the dynamic range relies on the use of partial resets of the integration node during integration time. The method described in the landmark paper “A 256×256 CMOS Imaging Array with Wide Dynamic Range Pixels and Column-Parallel Digital Out” (Decker et al., IEEE JSSC, 33(12), 1998, pp. 2081-2091) yields a good dynamic range extension, but suffers from high image lag and poor dark performance since an n+−p substrate photodiode is used as a sensing element. One could try replacing the n+−p substrate sensing element with a pinned diode and use the transfer gate to apply partial resets, but a straightforward application of this approach to the 4T pixel with a pinned photodiode does not lead to acceptable results in practice, since the contemporary technology has a lot of variance in the pinned photodiode depletion potential, pinned photodiode capacitance and also in the transfer gate barrier potential, what leads to unacceptably high fixed pattern noise of the high dynamic range image. Another possible alternative could be applying the partial reset signal to the floating diffusion node. In this case the temporal noise performance of the 4T pixel would be lost if the transfer gate is fully open. If the transfer gate is partially open (like in the approach presented in the above mentioned paper by Decker et al., the case where an overflow gate is used) so that a charge can leak from the photodiode to the floating diffusion, the temporal noise performance of the 4T pixel in low light can be preserved, but the variation of the potential at which the leakage occurs will introduce significant fixed pattern noise to the medium light and high light signal. The resulting uncertainties of the pixel response make the colour reconstruction especially difficult requiring per pixel calibration, which is usually not an acceptable solution for high volume embedded applications.
Several dynamic range extension methods have been proposed to tackle the high fixed pattern noise issue of the 4T pixel operating in dynamic range extension mode using a partial reset approach. The method described in U.S. Pat. No. 7,586,523 and in EP1924085 suggests a partial reset being applied to the photodiode potential well together with an additional partial readout, which will separately sample the signal above the partial reset level. The method described in U.S. Pat. No. 7,920,193 suggests a partial reset together with per-pixel knee point calibration. While the mentioned approaches reduce the impact of the pinned photodiode depletion potential variation and the transfer gate barrier potential variation, they have a drawback of applying partial resets to the photodiode potential well sacrificing part of the swing of the linear signal to the compressed signal. The available voltage range for partial reset is also limited, making it quite difficult in practice to achieve a high dynamic range extension. Another significant drawback of the mentioned approaches is the fact that contemporary technologies have high mismatch of the photodiode capacitance leading to high fixed pattern noise of the high dynamic range image.
Another drawback of the proposed approaches is a steep transition from linear signal to a highly compressed signal. As shown in “An adaptive multiple-reset CMOS wide dynamic range imager for automotive vision applications” (D. Hertel et al., IEEE Intelligent vehicles symposium 2008, June 2008, pp. 614-619) a custom tailored response function with gradual response slope change is highly advantageous in wide dynamic range imaging, the incremental signal to noise ratio should be optimized in the whole dynamic range to get the best detection capability of the high dynamic range imaging system. A simple one or two knee point response of the pixel does not satisfy the requirements of many applications like, for example, automotive driver assistance systems.
Hence, there is a need for a solution for operating a 4T pixel device which enables acquisition of high dynamic range scenes while maintaining a low fixed pattern noise level across the whole dynamic range and which preserves the 4T pinned diode pixel dark performance.